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I. INTRODUCTION 



This section presents an introduction 1;o the ^concept of 
impact, dissemination and the impact assessment design. 



it INTRODUCTION 

The primary purpose of funded vocatioi^^l education research and 
development activities is to improve vocational education.' A "good" 
program improvement process has not only solid reVearch, development 
and dissemination components, but also assesses the extent to which the 



whole process benefits some pr^etermined .public(s). Without impact, 
there is no program improvement. In order to determine if funded program 
improvement efforts really do make a difference, one must somehow identify 
or obtain indicators of the changes (e.g. skills^ knowledge, attitudes, 
behaviors. Interests, or perceptions) that occur within and among specific 
audiences. Because research and development activities are expensive, it 
is critical to assess their impact for accountability purposes and for 
making better judgments abojut further dissemination, identifying priorities 
and future funding activities. 

A 

Impact is a complex concept. The different types of impact which can 
be identified, along with numerous limitations and constraints, impinge 
upon attempts to define and assess impact. In defining impact and develop- 
ing an impact assessment design for the Research and Development Section/ 
Department of Adult, Vocational and Technical Education/Illinois State 
Board of Education, factors such as organization, management and the 
available human, fiscal and material resou^ces were taktn into consid- 
eration.^ Thefc:efore, the impact 'assessment design prese^ited herein is 
delimited according to the specific needs of the Research and Develop- 
ment S'ection as 'explained in the following sections. 

The Concept* of Impact 

Impact is defined as^'the consequence(s) of funded vocational edu- 
cation research and development projects ahd programs. A project is an 



activity funded by the Research and Development Section according to ^ 
Section 120, 130, 131 , 132, 133, 1 35 arid 136 of Public Law 94-482. A 
program is a group of projects that are seen by the Research and^ Develop- 
ment Section staff as having related goals. 

> 

The different types of impact a^rjd their relationship to each other 
are presented in Figure 1. As illustrated, one classification is by the . 
source of the impact. Somie impacts are the result of project activities. 
These are called project impacts . Other impacts are the result of a 
^ogram and are calls(fe program impacts . Program impacts are the result 
of the combined activities of all projects funded within a program. The 
comb^'neo activities are likely to create an impact synergy for the total 
program rather than having a simple cumulative effect. 

A second means of classifying impact is by time. 'Some impacts may 
be realized during cxi^ sdoh a'fter the completion of a funded activity. 
These are called (-hort term impacts . For the purpose of thi^ document, 
impacts which occur up^to 12 months after funding has ceased are considered 
short-term impacts. Other impacts may not occur .until 12 months or more 
after an activity' s ^ funding has ceased. These are called long term impacts , 

A third means of classifyingf impact is by intent (or lack of intent). 
In the planning stage of projects and programs- impacts are identified 
which are expected to occur. These are called intended impacts . Yet, 
oftentimes impacts occur which were not planned for or expected. These 
iiflpacts may be on the identified target audience or on 'persons, groups or 
agencies other than the target audience. These are called unintended 
impacts . 

Impacts may also be classified according to their relationship to 
goals. Some impacts may be a result of achieving the goals of the 

.\ . ■ 
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project or program. These are called direct impacts .. .If the impacts 
do not directly relate to the project goals but are perceived to be the 

result of project or program activities or products^ they are called 

I. 

indirect impacts . 

An impact can also be either positive or negative . While impacts 
must be positive for program im'provement to occur, one should not over- 
look negative consequences. If discovered early enough, negative impacts 
can lead to a change in project ^r program activities. This is turn 
could lead to improvejl project and program results or impacts. 

Delimiting the Problem 

As illustrated in Figure 1, impact is a complex concept "When con- 
^ sidering the numerous classifications of impact. The methodology for 
assessing impact of educational research and development is also at an 
infant stage. These two factors, combined with the need^ and resources . 
of the Research and Development Section necessitate delimiting the focus 
of the impact assessment design. 

The first consideration relates to the organizational structure for 
funding activities through the Research'and Development Section. At the 
present time, projects are funded on an individual basis, without a 
programmatic structure. Therefore, the impact assessment design for 
current implementation will focus only on project impacts. 

A second'^consideration is the degree to which impact is delineated 
according to the numerous classifications. For example, it would be 
possible to have a positive, direct, intended, short term, project 
impact. To describe impact by each classification would demand a high 
level of sophistication, both on the part of the impact assessors and 
the target audience. Since the impact assessment design presented 

ERJC 11 



herein is an initial attempt to provide direction and structure for 
assessing the impact of vocational education research and development 
in Illinois, it is delimited to include the classification up to and 
including short and long term impact. Figure 2 represents the portion 



Although long term impact must currently be assessed by projects, 
it is recommended that within 5 years assessment be done by program 
area. There are several advantages to working within the framework 
of programs when funding projects and assessing impact. 

1. A program structure can aSsist individual project directors 
to 'see that their project efforts contribute to an ov^erall 
plan and set of goals, thus tying their project goals and 
impact assessment to a larger whole. 

2. A program structure can also aid project directors to set 
more realistic and attainable expectations for/ their 
individual projects since they will be able to understand ^ 



how project efforts relate to program go^ls. ^ 

3. Assessing impacts of programs will help to eliminate the 
temptation to fund only low payoff, low risk projects because 
they have a high likelihood of showing a positive impact, in 
order to fulfill PL 94-482 impact requirements. Although high 
risk projects may have negative impacts, overall program impacts 
can still be positive. 

4. A program structure will encourage the planning for long term 
impact through well conceptualized, multiple-year efforts which 



allow sufficient time for research, development and dissemination 



of. impact assessment recoi 
111 inois. l 




d^d for immediate implementation in 



efforts. 



Alternatives for establishing programs within the ResArch and 
Development Section can be found in Appendix A. Once programs are 
established and program goals identified, program impacts afso can be 
identified. Although conceptually, the s^ort term impact of programs 
can cilso be assessed, it is recommended that short term ij^act assess- 
ment continue to focus on projects. Therefore, it is recommended that 
v/ithin five years, impact assessment should include the short term 
impact assessment of projects and the long term impact assessment of 
programs . At this point, long term program j'mpact assessment can re- 
place long term project impact assessment. Figure 3 represents the 
portion- of impact assessment which should be implemented within five 
years. ' 

Following implementation of short and long term proje9t and program 
impact assessment, it is possible to extend the design to include all 
classifications of impact. The more sophisticated delineation of impact 
may be used' in the assessment process, and/or in the interpretation of 
the findings. Figure 4 represents the final portion of the impact 
assessment' design which is recommended to be implemented within 7 years. 

Relationship of Dissemination to Impact 

It is important to note that in Illinois, dissemination of a 
project's product is an important, yet not exclusive step towa^'HG*^ 
achieving impact. Some states have defined impact as the 'extent of 

L 

dissemination, i.e., the number >f products dissemination. In Illinois, 
impact i|S considered to ibe a step beyond even the implementation level 
of dissemination. It is the consequence(s) of projects and programs * 
.which includes, but is not limited to dissemination activities. A 
brief^escription of the definition and process of dissemination. 




gure 3: TYPES OF IMPACT TO BE ASSESSED WITHIN FIVE YEARS 



lo 



\ 
/ 




Figure 4: TYPES OF IMPACT TO BE ASSESSED WITHIN SEVEN YEARS 
• > 
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utilized in Illinois, is provided to assist the reader in^ understxinding 

the j'elationship between dissemination and impaet. A more detailed 

explanation of dissemination may be found in A Systemmatic Model for 

DissenHnation (1981) and Dissemination Manual for Enhancing Program 

✓ ' • 

Improvement (1982), (Both documents are available from the Department 
of Adult, Vocational and jrechnfcal Education, Illinois State Board of 
Education, or through the Eric system). 

Dissemination of ' vocational education program improvement products 
is any activity which assists potential users to: 

1. Become aware of a product and its contents V 

2. Understand a product and how it can be used 

^ 3. Decide whether or not to adopt/adapt a product 

4. Impl ement and continue to use a product 

The concept of "use;" therefore, is J<ey in this definition of 

dissemination. The definition suggests "four levels pf assistance" 

(awareness, understanding, decision, implementation/use) that potential 

users of vocational education program improvement proBucts need prior 

' to adopting or adapting a product. The definition implies a progressive 

process which provides a structure for planning effective, dissemination 

activities. • ^ 

The systematic dissemation process in an important step toward 

iichieving impact. In Illinois, impact assessment reflects the conse- 

« 

. quences of all projects* activities, including the dissemination of the 
projects' products. Therefore, the likelihood of a project having a 
positive impact is increased through effective dissemination activities. 
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II. THE PROCESS OF ASSESSING IMPACT 

This section presents three designs: one for assessing the sht)rt 
term impact of research and development projects, the second for 
assessing the long term impact of research and development projects 
and the third for assessing program impact. ^ 



1/ 
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SHORT TERM PROJECT IMPACT 
AN OVERVIEW 



PURPOSE : ' 

To assess the short-term impacts of research apd development program Improvement 
projects to serve as a basis for making better judgnents about further 
dissemination and identifyirtg future priorities and activities to fund. 



SCOPE: 



Each year short term impact assessment will focus on 20 percent of all 
projects completed. 



A 



KEY QUESTIONS : 

1. How would you describe*the major impacts of this project?- 
1, t'ho was the target audience{s) most affected i)y the project's 
products! 

' 3. What dissemination activities were conductTed for thi^ project? 

4. What critical ingredients contributed to the achievement of impacts? 

5. What factors inhibited the achievement of impacts? 

6. Who are key individuals/groups and sites which could be contacted or 
visited to gain further evidence of the impacts of this project? 

7. What are the potential long term impacts of this project? 



INFORMATION SOURCES A)<D METHOD^ OF COLLECTION : 



1. 
2. 
3. 



Source 
Project Staff 



Methods of Collection 



1. 



Reports & Support- 
ing Proj. MatMs. 
Product Imple- 3. 
mentation Sites 



Interview 
Document Analysis 
On-Site Visits 



1. 
2. 
3. 



Person(s) Responsible 

Research and Development 
Section Staff 
Independent Impact 
Assessor(s ) 
Independerf^ Impact 
Assess 



penderft I 
5spr.(sh 



PROCEDURE: 



1. Reporting of impact data In project's final report. 

2. Selection of projects for short term impact* assessment. 

3. Interviews with project staff. 

4. Irwdepth investigation of projects* short term impact* 
AUDIENCE OF RESULTSr 

•Department of Adult, Vocational and Technical Education, Illinois State Board 
or Education* 

FORMAT OF RESULTS ; 

Written summary and recommendations* 
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SHORT T^RM PROJECT IMPACT ASSESSMENT 



^ J^URPOSE : 



< The purpose-of this design is to provide 
ment of the shQ.rt term impacts of research and 



guidelines for the assess- 
development program im- 
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provement projects. Short, term impact'^s defined as the consequence(s) 
of research and development progrjam improvement projects and is assessed 
within 12 months after the funded life of a prqject, .Short term impact 
assessment is a critica*l part of the program irrjprovement process in that 
it^an be used as a basis^for making better judigements about further dis- 
-semination of projects' products and for identifying priorities and future 
' activities to fund. Short term impact assessment also serves to satisfy 
accountability needs since impacts may occur during or immediately fol- 
lowing funding of a project and because short term tmpacts contribute 

/ 

to the long term impacts of a program area. ' 
SCOPE : . . ' 

Each year short term impact assessment will focus on the collection 
df impact assessment data fr©m 20 percent of the project^ completed 
within the year. 

KEY QUESTIONS : . ^ 

The following questions have been identified to serve as a basis for 
interviews with project staff and target audiences for the assessment of 
short term impact: 

1. How would you describe the major impacts of this project? 

1.1 What impacts were planned for dnd expected? 

K2 In what way did planned impacts change during 
the funded life of the project?^ 
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2. Who was the target^aud1ence(s ) most affected by the project's 
products? 

2.1 What impacts have occurred on the identified target .-^ 
audience(s)? . » 

2.2 What impacts have occurred on persons/groups other than 
the identified target audience(s)? 

3. What diss-emination activities were c^ducted for this project? 

3.1 What products were disseminated? 

3.2 How were th^ products disseminated? 

, 3.3 Who conducted the dissemination acti^ies? . j • 

3.4 How many products were disseminated? 

3.5 How has the dissemination of 'the project' s products affected 
the Impact of this, project? 

3.6 What type of continued disseminattbn of the project's 
primary products would, help to promote the impact of 
this project? 

4. What critical ingredients contributed to the achievement of impacts? 
^ For example* • 

4.1 What characteristics of the project staff contributed to impact? 

4.2 What characteristics of other key individuals or groups 
contributed to impact? 

4.3 What characteristics of the product contributed to impact? 

4.4 What characteristics of the setting contributed to impact? 

4.5 What aspects of the implementation process c ontributfed to 
impact? 

5. Who or what factors inhibited the achievement of impacts? 
For example: 

5.1 What characteristics of the project staff inhibited the 
achievement 'of impacts? 

5.2 What characteristics of other key individuals or groups 
inhibited the achievement of impacts? 

5.3 What^haracteristics of the product inhibited the achievement 
of imj^acts? 
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5.4 What characteristics of the setting inhibited the achievement 
of the impacts? * ♦ 

5.5 What aspects of the implementation process inhibited the 
achievement of fmpacts? . ^ , 

6. Who are key individuals/groups and sites which could be contacted 
or visited to gain further evidence of. the impacts of this project? 

^. What ape th6 potential long term impacts of this project? 

7.1 ' What future impacts are most likely to occur? 

7.2 Who might be affected by future impa'c't's? 

INFORMATION S OURCES AND METHODS OF COLLECTING 

See chart next page . 



(Short Term Project Impact) 
INFORMATION SOURCES AND METHODS OF COOtCTING 



Information 
* Source 


3>« 

• Method of 
Collection 


V 

Information Gained 


Person(s) 
Responsible 


^ Project staff 


Interview 


- History of project ancf product development 

- Background information on project staff 

- Project staff members" perceptions of 
project's impact ^ 

- Recommendation of issues, people and con- 
^sequences to De tol lowed up for in-depth 

assessment of short term impact 

- Collaboration (and/or discr^ancies) of 
information "gained from other 
sourcps/methods. 


Research and 
Development 
Section Staff 


2. Reoprts arid sup- 
^ ' porting project 
» materials 

a. Proposals 

b. Progress 

c. Final 
reports 

d. * Supporting 

material s 

■* # 


Document 
Analysis 


- Potential impact of the project * / 
The design and conceptualization of the 

^ products 

- The history of the project, including the 
source of support and the identity of the 
developers 

- The results and analyses of pilot and 
field testing of the product 

- Extent of diss'^iemination activities 

- Direct or indirect measures of actual impact 

- Recommendations for future contacts 

- Collaboration (and/or discrepancies) of 
information gairled from other sources/ 
methods 


Independent 

Impact 

Assessor(s) 


3/. Product 
* Implementation o 
Sites 

- Target 
audience 

- Personnel 
involved in 
impl ementation 

- Community and 
school members 

- Adaptations of 
' original 

ERIC P''^^"*^*^ 


On-site visits 

- Interviews 

- Observation 

- Document 
analysis 


- Additional input to above information 

- Description of actual impacts 

- Description of factors which contributed 
to the achievement of impacts 

- Description of factors which inhibited the 
achievement of impacts 

- CollaboratiQin (and/or discrepancies) of 
information gained from other sources/ 
methods 

/ 


Independent 

Impact 

Assessor(s) 

2.; 
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PROCEDURE ; 

The following procedure is recommended for the implementation of 
the short term project impact assessment design: 

1 . Reporting of impact related data in project's final report . The 
first step in the short term impact assessment is for the project 
Staff to provide information on impact in their final report. 
This includes: 

- any evidence of impact or progress toward impact, such as any 
baseline data collected, preliminary results from field work, 
effects of a workshop or impacts on project staff; 

- projections of long term impact; and 

- suggestions for gathering impact data. 

2. Selection of projects for short term impact assessment . The second 
step is the selection of 20 percent of all conpleted projects for short 
term impact assessment. It is recommended that a stratified random 
sample be selected by program area. This recommendation is based 

on the assumption that programs will be developed and utilized by 
the^Research and Development Section. If programs are not developed, 
then a random sample of all completed projects should be selected. 

3. Interviews with project staff . The third step in the short term 
impact assessment involves interviewing project staff to gain 
their perception of what impacts have occurred and to gain infor- 
mation regarding leads for further investigation of actual impacts. 

^This interview should be conducted during the last month of the 
final phase of funding so t*hat it will also help to provide closure 
to project activities. 

4. In-depth investigation of short term impact . The in-depth 
investigation should be completed within 'a year of the termination 
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of funding of the projects. It will involve the analysis of 
'^available reports and documents and on-site visits to product 
implementation sites. It is recommended that the general proce- 
dure for conducting an ort-site visit be a^pted from the Three 
Phase System for State Wide Evaluation of Vocational Education 
Programs (Evaluation Handbook for Team Leaders, Illinois State 
Board of Education, 3982). Generally the on-site visit would 
include: 

A. Pre-visitation activities 

a. Review of project reports and supporting materials 

b. Review of local agency materials 

c. Preparation of interview materials » 

B, On-site visit 

a. Interviews 

- Faculty 

- Administrative personnel 

- Students * 

- Community members^ 

b. Observation of classes/groups utilizing project products 
AUDIENCE qp RESULTS : 

The findings from the short term impact assessment should be 
directetl to the Department of Adult, Vocational and Technical Education, 
Illinois State Board of Education. 
FORMAT OF RESULT REPORTING : 

The findings of the short term impact assessment should be described 
in a v/r1tten report which would include sections on the major .impacts, dis- 
semination activities, factors which affected the achievement of impact and 
potential future impacts. 



SHORT TERM IMPACT ASSESSMENT MANAGEMENT PLAN 



Activity 


Steps to 
Achieve Completion 
of Activity 


Date of 
Initiation* 


• Date of 
Compl etion 


Personnel 
Involvement 


Intended Outcomes 


1 . Planning for 
intended 
impacts 


la. Write impact 
statements 

1 b. Revise impact 
statements 


1. During 
proposal 
writing 
stage 


1. End of 
funded 
life of 
. project 


la. Project 

staff 
lb. Assistance 

from RDS staff 

ff 


1 . Impact statements 
included 'in pro- 
ject proposal , 
progress reports 
and final report 


2. Reporting 
evidence of 
progress 
toward impact 


2a. Be aware of 
impacts or 
progress toward 
impact 
-2b> Collect impact --^ 
data 'when ap- 
propriate erg. . 
evaluation of 
workshop 
-2c /Report evidence 
of impact or 
^ progrjess toward 
impact 


2. Beginning 
of project 
life 


2. End of 
funded 
life of 
project 


2a. Project staff 
2b. Assistance 

from RDS staff 


2. Section on impact 
in the final 
J ' report 








3. Selection of 
projects for 
short term 
impact 
assessment 


3. Select random 
sample of 20% of 
completed 
projects 


3. flay 


3. June 


3. RDS staff 

\ 

A- 


3. List of projects 


4. Short term 
impact 
assessment 
interviews 


3a. Conduct inter- 
views with 
project staff 

3b. Report findings 
from interviews 


3. Last mo. 
of final 
phase of 
prpject 
funding 
(June) 


3. Last mo. 
of final 
phase of 
project 
funding 
(June) 


3a. RDS staff 
3b. Project staff 


3. Written summary 
of short term 
Impacts as an 
addendum to final 
report 
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SHORT TERM IMPACT ASSESSMENT PLAN -continued 



Activity 


Steps to 
Achieve Completion 
of Activity 


Date of 
Initiation 


I Date of> 
^mpl etion 


Personnel 
Involvement 


% 

Intended Outcomes 


5. In-depth in- 
vestigation 
of short term 
impact 


4a. Review available 

documents for 

impact related 

information 
4b. Plan on-site 

visits 
4c. Conduct on-site 

visits 
4d. Conduct addition-. 

al interviews as 

needed 


4a. July 

4b. July 

4c. September 

4d. September 


4a\july 

4b. August 
4c. Hay 
4d. May 


4a. Independent 
Impact Assessor 
(IIA) 

4b. IIA 

4c. IIA and target 

audience 
4d. IIA ^ 


4a. Impact data 

4b. Assessment plan 
4c. Impact data 
4d. Impact data 


6. •'Prepare short 
term impact 
report 1 


5. Write report 


5. June 


5. June 


5. IIA 
» 


5. Written report 



*all dates are based on the fiscal year of July 1 to June 30. 
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LONG TERM PROJECT IMPACT 
• AN OVERVIEW 



PURPOSE; 



To assess the long term lMp«cts of research and developnent projects to satisfy 
accountability needs- and to serve as a basis for Identifying priorities and 
future activities to fund. 

SCOPE : ^ 

Each year, long term project Impact assessment will focus on 10 percent of the 
projects cofljpleted two to' five years prior to assessment. 

KEY QUESTIONS : 

1. How would you describe the major Impacts of this project? 

2. Who was the target aud1ence{s) .most affected by the project's product? 

3. 'What dissemination activities were conducted after project funding cease^? 

4. ^hat critical Ingredients contributed to the achievement of Impacts? 
5« What factors Inhibited the achievement of Impacts? 

6« Is there reason to believe that additional Impacts will occur In the future? 

INFOfWATIOH SOURCES AND METHODS OF COLLECTION : 
J ^ 



Source 



Methods of Collection Person(s) Responsible 



1. 
2. 



3. 



' Research and Development 1. Interview 
Sfectlon Staff 

Reports 4 Supporting 2. 
Project Materials 3. 
Product Implementation Sites 



Document Analysis 
On-site Visits 



1 . Independent Impact 
Assessor(s) (IIA) 

2. IIA 

3. IIA 



PROCEDURE: 



1. Selection of projects for long term Impact assessment. 

2. Interviews with Research and Development Section staff. 

3. In-depth Investigation of long term project Impact. 

AUDIENCE OF RESUUS : 

Department of Adult, Vocational and Technical Education, Illinois State Board 
of Education. 

FORMAT OF RESULTS : 

Written summary and recommendations. 
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LONG TERM PROJECT IMPACT ASSESSMENT 

PURPOSE : 

The purpose of this design is to provide guidelines for the 
assessment of long term impacts of research and development projects. 
Long term project impact is* defined* as the consequence(s) of research 
and development projects assessed 12 months or more after the termi- 
nation of funding. Long term impact assessmen-t can help determine if 
vocational education research and development activities improve voca- 
tional education programs, thus satisfying accountability need6. Long 
term impact assessment can also serve as a basis for identifying 
priorities and future funding activities. 
SCOPE: 

Each year long term project impact assessment will focus on an in- 
depth investigation of 10 percent of projects completed two to five years 
prior to the as,sessment. 
KEY QUESTIONS r 

The following questions have been identified to serve as a basis for 
the collection of long term project impact information: 

1. How would you describe the major impacts of this project? 

1.1 What impacts were planned for and expected? 

1.2 In what way did planned impacts change during continued 
use of project's product? ^ 

2. Who was the target audience(s) most affected by the project's product? 

2.1 What impac^ have occurred on the Identified target audience(s)? 

2.2 ^hat impacts have occurred on persons/groups other than the 
identified target audience(s)? 

32 



3. What dissemination activities were conducted after project funding 
ceased? 

3.1 What products were disseminated? 

3.2 How were the products disseminated? 

3.3 Who conducted the dissemi^^ation activities? 

3.4 How have dissemination activities affected the impact of the 
project? 

* - 

4. What critical ingredients contributed to the achievement of impacts? 
For example: 

4J What characteristics of the product contributed to impact? 

4.2 What characteristics of the setting contributed to impact? 

4.3 What aspects of the implementation process contributed to 
impact? 

4»4 How did key individuals/groups contribute 'to impact? 

5. What factors inhibited the achievement of impacts? For example: 

5.1 What characteristics of the product inhibited the achievement 
of impacts? 

/ 5.2 What characteristics of the setting inhibited the achievement 
of impacts? 

•'5.3 What aspects of the implementation process inhibited the 
achievement of impacts? 

5.4 How did individuals/groups inhibit the achievement of impacts? 

6. »Is there reason to believe that additional impacts will occur in 
the future? , ' 

6.1 What future impacts Vay occur ? 

6.2 Who might be affected by future impacts? 
INFORMATI ON SOURCES AND METHODS OF COLLECTING 

^ 

See chart next page 
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, (Long Term Project Impact) / 



INFORMATION SOURCES AND METHODS OF COLLECTING 



(Information 
Source^ . 


Method of 
Collection 


\ 

Information Gained 


Person(s) 
Responsible 


1 . Research and 
Development 
Section Staff 

* 


Interview 


- History of project and product development 

- Research and Development Section staff 
members' perceptions of the impacts of the 
project 

- Dissemination activities conducted after 
the termination of project's funding 

- Recommendations of issues, people and 
consequences to be followed-up ^ 

- Collaboration (and/or discrepencies) of 
information gained from other sources/ 
methods 


Independent 

Impact 

AssessorCs) 


2. Reports & Sup - 
porting Project 
Material s 

a. Final report 
of project 

b. Short term im- 
pact report (if 
available- for 
project) 

c. Supporting 
Project Materials 


Document 
Analysis 

K 


• • 

- Potential impacts of the project 

- Design and conceptualization of the products 

- Identity of the devei^opers \ 

- Extent of dissemination activities 

- Project staff members' perceptions of the 
project's short term impact 

- Evidence of short term impacts 

- Factors which contributed to or inhibited 
the- achievements of impacts 

- Collaboration (and/or discrepencies) of infor- 
mation* gained from other sources/methods 


Independent 

Impact 

Assessor(s) 


3. Product 

Impl ementation 
Sites 

- Target 
audience 

- Personnel 
involved in 
impl ementation 

- Community and 
school members 

-^Adaptations of 
project's products 


On-site 
Visits 

- Interview? 

- Observation 

- Document 
Analysis 

0 


- Additional input to above information 

- Description of impacts 

- Factors whfch contributed to or inhibited 
^the achievements of impacts 

- Collaboration (and/or discrepencies) of infor- 
mation gained from other sources/methods 


. Independent 
Impact 
AssessorCs) 
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PROCEDURE : 

The following procedure is recommended for the implementation of the 
long ternPproject impact assessment design: 

1. Selection of projects for long term impact assessment . The first 
step is to select the projects for long ferm impact assessment. It 
is recommended that a random sample be selected from a population of 
/ projects completed two to five years prior to the assessment. Ten^ 
percent of the population would provide a manageable number of 
projects, although the percentage could be varied depending on the ^ 
population 'size. Once the random sample has been chosen it is recom- 
mended that the list'of projects be submitted to the Research and 
Development Section staff for review and approval. It is possible 
that a strong rationale exists for the inclusion or exclusion of a 
specific project, and the impact assessment design should allow 
such flexibility. 

Interviews with Research and Development Section Staff , The second 
step is to interview the contract administrator responsible for each 
project selected. The contract administrator should be able to 
' provide: 1) a brief history of the project and^ product developnrent , 



2) his/her (5erception of the impacts of the project, 3) the infor- 
mation on the dissemination activities conduc^ted after the termination 



of project funding, and 4) recorranendations of issues, people and 
consequences to be followed-up. 
3. In-depth investigation of long term impact . The in-depth inves- 
tigation of long term project impact involves the analysis of 
available reports and documents, and on-site visits to .product 
implementation sites. It is recommended that the procedure for 

r 
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conducting the on-site visits be adajj|^rfroiiifffe Three Phase System 
for State-Wide Evaluation of Vocational Education |:^grains (Evalu- 
ation Handbook for Team Leaders, Illinois State BoardW^ Education, 
1982)'.- Generally the on-site visits would include: ^ 

A. Pre-visitation activities 

a. Review of project reports and supporting project materials 

b. Review of short term impact reports 

c. Review of local agency materials 

d. Preparation- of interview materials 

B. On-site visits 

, a- Interviews ^ 

- Faculty 

• /Administrative personnel 
^ Students 

- Community members 

b. Observation of classes/groups utilizing projects' products 

AUDI€NCE OF RESULTS ; 

The findings from the long term impact assessment should be directed 
to the Department)of^Adult, Vocational and Technical^Education, Illinois 
State Board of Education. 
FORMAT OF RESULT REPORTING ; 

The findings of the long term impact assessment should be described 
in a written report which would include sections on the major impacts, dis 
semination activities, factors which affected the a^chievement of impact 
and potential future impacts. 
MANAGEMENT PLAN: See chart next page. 
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LONG TERM PROJECT IMPACT ASSESSMENT MANAGEMENT PLAN 



Activity 


Steps to 
Achieve Completion 
Of Activity 


Date of 
Initiation* 


Date of 
Completion* 


Personnel 
ImtaJ vement 


Intended Outcomes 


1 • Sel ection of 
projects for ' 

1 nnn f prm 

1 1 U M \J U CI III 

impact 
assessment 


1 . Select random 
sample of 10^ of 

nro i pct^ rnm- 

^1 V^JCV^l^O V^V^III 

pleted 2-5 
years prior to 
assessment 


1. July 


i. July 


1. RDS^taff 


1 . t)ist of projects 


Z. Document 
analysis 


2. Review documents 
containing impact 
data such as 

report and short 
term impact report 
(if available) 


2. July 


2. August 


2. Independent 
Impact 

Assessor (IIA) 


2. Impact data 


3. Long term impact 
assessment 

1 1 1 L CI V i cWo 


3a. Conduct inter- 
views with. RDS' 

O La 1 1 

3b. Report findings 
from interviews 


3. August 

7 


3. Sept. 


IIA and RDS 


3. Impact data' and 
information on 

^ persons/agencies 
to contact for 
further inves- 
tigation 


. 4. In depth in- 
vestigation 
of long term 
impact 


4a. Select locations 
for on-site visits 

4b. Plan on-site 
visits 

4c. Conduct on-site 

V 1 b 1 L b 

4d. Conduct addi- 
tional inter- 
views as needed 


4a. Sept. 
4b. Sept» 
4c. Oct. 
4d. Oct 


4a. Sept. 
4b. Oct. 
4c. Mcly 
3d. May 


4a. IIA 
4b. IIA 
4c. IIA 
4d. IIA 


4a. List of 
locations 

4b. Impact Assess- 
ment plan 

4c. Impact data 

4d. Impact data 


5. Prepare long 
term impact 
report 


5a. Anat^^ze and 
synthesize 
findings 

5b. Write report 


5. June 


5. June 


5. IIA 


5. Written report 



.,*These dates are based on the fiscal year of July^ 1 to June 30. . 
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LONG TERM PRpGRAM IMPACT 
AN OVERVIEW 



PURPOSE; 



To assess the long term Impacts of research and development programs to satisfy 
accountability needs and to serve as a basis for Identifying prior'itles and 
future activities to fund. 

SCOPE ; 

Each year long term program Impact assessment will focus on 20 percent of the 
program areas. 

KEY QUESTIONS ; 

1. How would you describe the major Impacts of this program? 

2. Uho was the target aud1ence{s) most affected by this program? 

3. What dIssaMnatlon activities were conducted for projects' products 
within the program? 

4. What critical Ingredients contributed to the achievement of program 
Impacts? 

5. What factors Inhibited the achievement of program Impacts? 

6. Is there reason to believe that additional program Impacts will occur 
In the future? 



INFORMATION SOURCES AND METHODS OF COLLECTION : 

Source Methods ,of Collection 



1. 


Research and* Development 


1. 


Interview 


1 . 




Section staff 








2. 


Reports & Supporting 


2. 


Document Analysis 


2. 




Program Materials 








3. 


Product Impl Mentation 


3. 


On-site Visits 


3. 




Sites 




Program Specific 










Survey 





Person(s) Responsible 

Independent Impact 
Assessor(s) (IIA) 
IIA 

3. IIA 



PROCEDURES ; 

U Selection of program{s) for Impact assessment. 

2. Document analysis of existing reports and support materials. 

3. .Interviews with Research and Development Section staff. 

4. On-site visits to product Implementation sites. 

AUDIENCE OF RESULTS ;' 

Department of Adult, Vocational and Technical Education, Illinois State Board 
of Education. 

FORMAT OF RESULTS : 

Written summary* 
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LONG TERM -PROGRAM IMPACT ASSESSMENT 
(To replace long term project impact -assessment once programs have been developed) 

VURPOSE : N . ^ . 

' Bbe purpose of this design is to provide guidelines for the assessment * 

- c 

of long term impacts of research and "development programs. Program impact is 
defined as the consequence(s) of research and development programs. A program 
is a group^of program improvement' projects thit have related goal s. ^ Therefore, 
the impacts of projects contribute to the impacts of a program. Program 
impact assessment can help determine if vocational education research and 
development activities improve vocatiohal%^education programs, thus satis- 

* i 

fying accountability needs. Program, impact assessment can also serve as a 
^sis for identifying priorities and future activities to fund. ^ , ^ 
SCOPE : ' ^ • 

Each year long term program impact assessment will. focus on an in-depth 
''investigation of 20. percent of identified program areas. By assessing the 
impacts of 20 percent of the programs, each program wil 1 be "assessed every 
five years. . % 
KEY QUESTIONS : ' 

The following questions have been, identified to serve as a basis for 
♦ th6 collection of program impact information; 
1.^ How ivpyld you describe the major impacts of this program? 



1.1 What impacts were planned for and expected as a result 
of this program? ^ 



1.2 What impacts were planned for and expected as" a result of 
projects within this program which contribute to program 
impacts? 

1.3 What were the actual impacts? 

2. Who was the target audience(s) most affected by the program? 
2.1 What impacts have occurred on the target audience(s)? 
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2.2 -What^mpacts' have occurred on persons/groups other than 
the target audience(s)? 

3/ What dissemination activities were conducted for projects' 

products within the program? 

3.1 What products were disseminated? 
• 3.2 How were. the products disseminated? 

3.3 Who conducted dissemination activities? y 

3.4 have disserni^nation activities affected the impact of 
the program? 

4. What critical ingredients contributed to the a:hieveinent of 
program impacts? 
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4.1 In what way were specific projects within the program critical 
to the achievement of program impacts? , 

4 2 In what way was -frke interrelationship between a nd/or among 
projects within theVograTn critical to the a.chievement of 
program impacts? 

4.3 In what way were human, material and/or fiscal resources 
critical to the achievement of program impacts? 

5. What factors inhibited the achievement of program impacts? 

5.1 In what way did specific pnojects within the program 
inhibit the"~ achievement oj^ program impacts? 

5.2 In what way did the interrelationship bet ween and/or among 
projects within the program inhibit the achievement of 
program imparts? 

5.3 In what way did human, material and/or fiscal resources 
inhibi't the achievement of program impacts? 

6. Is there reason to believe that additional program impacts will 
occur in the future? . , ^ 

6.1 What future impacts may occur? 

6.2 Who might be affected by future impacts^ ^ 
INFORMATION SOURCES AND METHODS OF COLLECTING 

See chart next page. 



42 



.INFORMATION SOURCES AND METHODS OF COLLECTING 



Information 
Source 


Method of 
Collection . 


Information Gained . 


Person(s) 
Responsible 


1 . Research and 
Development 
Section staff 


Interview 


- History of program, including project and product development 

- Research and Development Section staff members' perceptions 
of tjie impacts of the program includitig projects within the 
program ^ 

- Dissemination activities conducted after the termination of 
projects' funding 

- Program goals and potential long term impacts of the program 

- Recommendations of issues, people and consequences to be 
fol lowed-up S 

- Collaboration (and/or discrepancies) of information gained 
^ from other sources/methods 


Independent 
Impact 
Assessor (s) 


2. Reports and sup- 
porting mat'ls ^ 

a. Final reports 
of projects 
within the 
program 

b. Short term 
impact 
reports (if 
available) 
Supporting 
Materials 


Document 
Analysis 


- Potential impacts of the projects within the program "^N^ 

- Design and conceptualization of the products 

- Extent of dissemination activities * 

- Project staff members' perceptions of the short term impact 
of projects within the program 

- Evidence of short term impacts of projects within the program 

- Factors which coFjtributed. to or inhibited the achievements - 
of short term project impapts 

- Evidence of short term project impacts' contributions to 
long tenn program 'impacts 

- Collaboration (and/or discrepancies) of information gained 
from other sources/methods ^ - 


Independent 
Impact 
Assessor (s) 

« 


3. Product 

Inipl ementation 
Sites 

- Target audience 

- Personnel in- 
. voTved in 

irapl errtent-ation 

- Community and 
school members 

- Adaptations 
of products 


On-site 
Visits 

- Interviews 

- Observation 

- Document 
Analysis 

* 


- Addit>onal input to above information 

- Description of program 'impacts ^ 

- Factors which contributed to or inhibited the afchievements 
of program MpStfes 

- Collaborat/ion (andy^ discrepancies) of irjformation gained 
from other sources/methods |^ 


Independent 

Impact 

Assessor(s) 

^• • 


4. Additional 
product impl e- 
mentation sites 


Program 
Specific . 
Survefy 


- Additional program impact data ^ 

- Collaboration (and/or discrepancies; of information gained 
from other sourc^s/mfethods 


Independent 

T m f*^ ^ 

impdc L 

Assessor(s) 
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PROCEDURE : 

The following procedure is recommended for the implementation of the 
long term {)rogram tinpact assessment design: 

1. Selection of program for impact assessment . It is recommended that 
the Research and Development Section s^^^ assume responsibility for 
the selection of the program to be assessed. Twenty percent of the 
programs should be selected so that each program will be assessed 
approximately every five years. 

2. Document analysis of existing reports . The second step ^jn^he 
program impact assessment involves a review and analysis of existing 
reports, such as: program planning reports, final reports of projects, 
and the short term impact reports, if available. Information gained 
from these retports will include:* program goals and intended impacts; 
goals, objectives, and intended impacts of projects within the progr^^j^ 
target audience of project products, implementation sites, and 
evidence of short term impacts. 

3. I nterviews with Research and Development Section staff . The third 
step is to interview the contract administrators responsible for .the 
projects within the program. The contract administrators should be 
able to provide information concerning their perception of the role 
that their project played in program impact, as well as relevant 

, activities or dissemination efforts which-have occurred since the ^ 

' termination of project funding. 

4. On-site visits to a selected sample of sites where project outputs 
have been implemented . J3n-si*te visits should be made to im'plemen-y 
tation sites of at least three projects within the program. Sites 
.should hQ chosen on a random basis. It is recommended that the 
procedure for conducting the on-site visits be adapted from the 



34 



Three Phase System for State-Wide Evaluation of Vocational Education 
Program (Evaluation Handbook for Team Leaders, Illinois State Board 
of Education, 1982).. Generally the on-site visits would include: 
A. Pre-visitation activities 

a. Review of project reports and supporting project and ^ 
program material s 

b. Review of short term project impact reports 

c. Review of local agency materials 

d. Preparation of interview^materials 



B. On-site visits 

a. Interviews 

- Faculty 

- Administrative Personnel 

- Students 

- Community Members * 

b. Observation of classes/groups utilizing projects' products 
5. Development and ^Administration of a Program Specific Impa^ct Survey . 

^ Using the information gained during the on-site visits,* a program 
specific impact survey can be developed. This survey can be adminis- 
tered to target audiences of'additional implementation sites to gather 
additional impact data and to validate the findings from the on-site 
visits. A sampl e^ program-specif icJunp^^ct^rvey can be found in 
Appendix B. 

AUDIENCE OF RESULTS : 

The findings from the program impact assessment should be directed to 

the Department of Adult, Vocational and Technical Education, Illinois State 

Board of Education. 
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FORMAT OF RESULT REPORTING : 

The findings of the program impact assessment should be described 
in a written report which would include sections on the major impacts, 
, dissemination activities, factors which affected the achievement of 
impacts and potential future impacts. * 
MANAGEMENT PLAN : . • 

See chart next page. 
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PROGRAM IMPACT ^ASSESSMENT MANAGEMENT PLAN 



Activity 


Steps to 
Achieve Completion 
of Activity 


Date of 
Initiation* 


Date 
Completion* 


Personnel 
Involvement 


Intended Outcomes 


1 • Sel ection of 
programs for 
impact 
assessment 


1 . Select 20% of 
programs 


1. July 


1. July 


1. RDS staff 


1. Program{s) name 


2» Document 
analysis 


2. Review documents 
containing impact 
data ^urh 
projects' finals 
reports and short 
term impact 
report 


2. July 
# 


2. August 


2. Independent 
Impact 

flccoccnr' iTTA^ 


2. Impact data 


3. Impact 

a s spssmpnt 
^ interviews 


3a.. Conduct interviews 
with RDS staff 

WIUII f\U*J Obull 

3b. Report findings 
from interviews 


3. August 


3. Sep^. 


3. IIA an^ RDS 


3. Impact data and 

llliUiNlaLIUll Ull 

persons/agencies 
to contact for 
further 
investigation 


4. In--depth 
investigation 
of program 

imnart 

1 III V« l« 

\ 


42^. Select locatipns 

for on-site visits 
4b. Plan on-site visits 

dr Prinrliirf nn — ct + o . 

visits ' 
4d. Conduct additional 
interviews as 
needed 


4a,. ^pt. 
4b. Sept. 
4d. Oct. 


4a. Sept. / 
4b. Oct. 

^c * ria 1 cii 

4d. March 


4a. IIA 
4b. IIA 
4d. IIA 

• 


4a. Lis^t of 

locations 
4h. Impact Assessment 

Hp q "! nn 

UC o 1 Ull 

4c. Impact data 
4d. Impact data 


5. Develop program 
specific impact 
survey 


5. Develop survey 
* 


5. March 


5. March 

,f 


5. IIA 


5. Survey 



^continued 



PROGRAM IMPACT ASSESSMENT MANAGEMENT PLAN (continued) 



Activity 

> 


Steps to 
Achieve Completion 
of Activity 


Date of 
Initiation* 


Date of 
Compl etion* 


Personnel 
Involvement 


Intended Outcomes 


6. Administer 
survey 


6a, Sel ect sftes to 
survey ^ 

6b. Conduct survey 

6c. Analysis of 
findings 

6d. Report findings 


6.. April 

— f- 


6. May 


6. IIA 


6. Impact data 

- * 


7. Prepare long 
term program 
impact report 


7a. Analyze and 

synthesize 

findings 
7b. Report findings 


7. June 


7. June 


7. IIA 


7j Written report 



*These dates are based on the fiscal year of July 1 to June 30. 
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III. MANAGING IMPACT ASSESSMENT 

This section describes the roles and responsibilities of project 
staff. Research and Development Section staff,* and the. independent 
impact assessor(s). It also describes alternatives for who might 
assume the role of the Independent impact assessor. 
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MANAGING IMPACT ASSESSMENT 

Assessing the short and long term impacts of research and development 
program improvement projects and programs involves many individuals and/or 
groups. To effectively manage this process, each group must be awane of 
their role ancL.responsibil,ities and how these responsibilities fit into 
the process. This section describes the roles and responsibilities of 
project staff. Research and Development Section staff and the independent 
impact assessor(s). It also describes alternatives for who might assume 
the role of the independent impact assessor .^.depending on available 
resources. 

. ROLES AND RESPONSIBILITIES RELATED TO IMPACT ASSESSMENT 
PROJECT STAFF: 

Project staff become involved with impact assessment as they write, a 
proposal and describe the potential impact of the .project. The Respon- 
sibilities of project staff throughout the funded life of a project are 
primarily to identify potential impacts an'd to report progress toward 
actual impact. More specifically, responsibilities of project staff 
incl ude: 

1. Developing impact statements to be included in the proposal for 
each phase of ^the proje ct; 

2. Revising the impact statements and reporting progress toward 
the achievement of impact in the progress reports to the 
Research and Development Section staff; 

3. Reporting dissemination efforts and any dissemination follow-up 
I information in their final reports; 
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4. Providing information on impact in their final report which includes: 

A. , Any evidence of impact or progress toward impact, such as any 

baseline data collected, preliminary results from field work, 
effects of a wof^kshop or impacts on project staff; 

B. Projections of long term impacti 

C. Suggestions for gathering data; and 

5e Meeting with contract administrators to provide informat-ion regarding 
their perception of the project's impact and to recommend issues, 
people and consequences that should be followed-up. 

RESEARCH AND DEVELOPMENT SECTION STAFF : 

Jhe responsibilities of the Research and Development Section 

Manager include: 

1. Identifying research and development programs and program goals as 
a basis for project funding decisions and for the development of 
program impact statements; 

2. Determining who will assume the role of the independent impact 
assessor(s), based upon an assessment of available resources; 

3. Assigning responsibilities to those persons involved in the impact 
assessment process; and 

4. Supervising the impact assessment process. 
J^he-responsibil ities of the Research and Development Section staff 

include: 

K , Assisting project staff in developing, revising and reporting 

impact statements; 
2. Assisting project staff in developing, conducting and reporting 

dissemination efforts; 
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3. Assisting project staff in identifying and reporting evidence of 
impact or progress toward impact; 

4. Conducting 1nterviews^1?iih staff of those projects selected for 
short terjTuimpact assessment to obtain their perception of the 
project's impact; 

5. Bji^Jnter viewed by the independent impact assessor for the 
long term project or program impact assessment; and 

6. Assuming additional responsibilities if the independent impact 
assessor(s)'s responsibilities are divided among the Research 
and Development Section staff. 

INDEPENDENT IMPACT ASSESSOR(S) : 

The responsibilities of the independent impact assessor(s) include: . 

1. Reviewing available documents which include impact information, such 
as finiil reports or v/rite-u|>^f-nnlerviews with project staff; 

2. Interviewing Research and Development , Section staff regarding the 
impact of projects they have monitoredj^ ^ 

3. Conducting on-site visits -to locations where implementation of 
project products have occurred; 

A. Developing and <r6hducting a program specific survey on program 

i^cts; and \^ 
5. Writing a summary report on sKbrt term project impacts and long 

tdrm project or program impact. ^ 

INDEf^NDENT IMPACT ASSESSOR(S) ALTERNATIVES 
The roles and responsibilities of the independent impact assessor(s) 
may be assumed by different people/agencies. The decision of who is to , 
assume these responsibilities will most likely be dependent on available^ 
resources of both time and money.' Three alternatives have been identified 
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Alternative #1 : 

The first alternative is to contract with an external group/agency 
tb conduct the impact assessment. One means of doing this would be 
through the "Request for Proposal Process:" Advantages to this alter- 
natJ^ve are that few additional responsibilities would be placed on the 
Research and Development- Section staff and a greater degree of objectivity 
will be achieved through the use of an external group. A disadvantage 
might te the higher cost. If this alternative is chosen', it is recom- 
mended that the impact assessment project be "monitored" by the manager 
of the Research and Development Section or a staff member designated to 
coordinate the impact assessment in order to provide continuity from one 
year to the next. 
Alternative J2 : 

A second alternative would be to utilize staff from another unit 
withfti the state agency, ^ch as the Program Evaluation and Assessment 
Section within the Division of Planning Research and Evaluation. 
As in alternative #1, this alternative would place few additional respon- 
sibilities on the (Research and Development Section staff and would provide 
for objectivity. In addition, it would promote cooperation and collabora- 
tion between sections. The feasibility of this alternative would depend 
on whether staff from another section would have time and expertise to 
assume such responsibilities and If the mechanics of achieving collabora- 
tion could be achieved.' 
Alternative ^3 : 

The third alternative is to divide the responsibilities of the impact 
assessor among the Research and Development Section staff. It is recom- 
mended that one staff member be identified to facilitate the impact 
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assessment in order to coordinate the efforts of all staff members. An 
advantage to this alternative is the low cost, although the feasibility 
would depend on whether staff members would have time to assume such , 
responsibilities. A disadvantage is the bias of staff members due to 
previous involvement in project activities. If this alternative is 
chosen, it is recommended that the number of projects chosen for in- 
depth assessment be equivalent to the number of staff members assuming 
responsibilities. Division of responsibilities may be as follows: 
SHORT TERM PROJECT IMPACT ASSESSMENT 

« 

Joint responsibilities of staff: 

- Selection of projects for in-depth assessment. 

- Assignment of projects to individual staff members. 

- Discussion concerning findings of on-site visits. 

- *Compilation of the findings of the short term impac^ assessment. 

Responsibilities of individual staff members: 

# **" 

- Exit interviews with staff of completed projects (number will 
vary depending on how many projects are chosen for short term 
impact assessment. 

- Conducting the in-depth investigation of short term impact of 
one proj'ect. This study would include a review of available 
reports and on-site visits. 

- Writing a report summarizing the findings. 
LONG TERM PROJECT' IMPACT ASSESSMENT 

Joint responsibilities of staff: 

- Selection of projects for impact assessment. 

- Selection of implementation sites for on-site visits. 

- Assignment of implementation sites to individual staff members 
(one location per person). 

- Discussion concerning findings of on-site visits. 
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- *Compnation of findings. , 

» 

Responsibilities of individual staff members : 

- Review of available documents. 

- On-site, visit to one implementation site. 
LONG TERM PROGRAM IMPACT ASSESSMENT 

Joint responsibilities of staff: 



- Selection of program(s) for impact assessment. 

- Selection of implementation sites for on-site visits. 

- Assignment of implementation sites to individual staff members. 

- Discussion concerning findings of on-site visits.* 

- ^Development and administration of program-specific survey 
(if determined to be desirable). 

- *Compilation of findings. ^ » 
Responsibilities of individual staff members : 

- Review of available documents. 

- On-site visits to implementation sites. - 



*These responsibilities ma-y be assigned to the impact assessment 
specialist if a staff member is designated as such. 
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IV. UTILIZING IMPACT ASSESSMENT DATA 



This section provides possible uses of short term and long term 
impact assessment data and is organized by the key questions 
identified in the impact assessment design. 
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USE OF SHORT TERM IMPACT DATA 



Key Questions 



Use of Information Collected 



E 



1. How woyld you describe the major 
impacts of this project? 

2. Who was the target audience(s) 
most affected by the project's 
product? 



Discussions related to short term 
planning of research and develop- 
ment program improvement efforts 
including: 

a. priority setting 

b. future funding of research 
and development projects 

Accountability data for funding 
agency and policy makers 
Public relations within a local 
community 



c 
c 



E2J 

J) 



What dissemination activities 
were conducted for this 
project? 



Decisions related to: 

a. promotion of a project's product 

b. funding for continued 
dissemination of a 
project's product 

c. need for revisions or 
changes in a project's 
product 

d. effect of dissemination on 
impact 



4. What critical ingredients con- 
tributed to the achievement of 
impacts? 

5. What factors inhibited the 
achievement of impacts? 



Identification of common elements 
in the development and imple- 
mentation of project'? products 
which contributed to or inhibited 
impact. This- may ^include 
information on: 

a. project staff 

b. processes 

c. resources of time and money 
Application of knowledge gained 
to future funding efforts 



6. Who are the key individuals and 
sites which could be contacted 
to gain further evidence of the 
impacts of this project? 



Continued investigation or. 
follow-up of short term impact 
Potential leads for long term 
impact assessment 
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Key Questions (continued) 



What are the potential long term 
impacts of this project? 



Use of Information Collected 



Decisions related to long term 
planning of research and 
development program improvement 
activities 

Contributions to lonn term 
impact assessment 
Accountatkil ity purposes 
Public relations within a 
local community 
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USE OF LONG TERf-1 PROJECT IMPACT DATA 



Key Questions' 



Use of Information Collected 



1. ''How would you describe the 

major impacts? 

2. Who was the target audience 
most affected by the project? 



Decisions related to fong term 
planning of research and 
development program improvement 
efforts, including: 

a. priority setting 

b. future funding of research 
and development projects 

Accountability data for funding 
agency and policy makers 
Public relations within a local 
conwiunity 



3. What dissemination activities 
were conducted? 



Evaluation of dissemination 
Identification of successful 
dissemination processes r 
Decisions related to the need 
for continued dissemination 
of products 

The effect of dissemination on 
impact 
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4. What critical ingredients 
contributed to the achievement 
of Impacts? 

5. What factors inhibited the 
achievement of impacts? 



6. 



o • 

CL 



Is there reason to believe that 
additional impacts^ will occur 
in the future 



- Identification of key elements 
• which contribute to or inhibit 

the achievement of impact 

- Application of knov;ledge gained 
to future funding efforts 



Decisions relating to long term 
planning of research and 
development program improvement 
efforts 
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USE OF LONG TERM PROGRAM IMPACT DATA 



Key Questions 



Use of Information Collected 



f 

K How would you describe the 

major impacts? 
2. Who was the target audience 

most affected by the program? 

• 

* 


- Decisions related to long term 
planning of research and 
development program improvement 
efforts, incl uding : 

a, priority setting 

b. future funding of research ' 
and development projects 

- Accountability data for funding 
agency and policy makers 

- Public relations within a local 
community 


3, What dissemination activities 
j were conducted? 

9 

i 

1 

1 

! • 

1 


- Evaluation of dissemination 

- Identification of successful 
dissemination processes 

- Decisions related to the need 
for continued dissemination 
of products 

- The effect of dissemination on 
impact 


f 

, 4. What critical ingredients 

contributed to the achievement 
, of impacts? 
1 5* What factors inhibited the 
\ achievement of impacts? 


- Identification of key elements 
which contribute to or inhibit 
the achievement of impact 

- Application of knowledge gained 
to future funding efforts 


1 

1 6. Is there reason to believe that 

additional impacts will occur 
1 ' in the future? 

; 


- Decisions relating to long range 
planning of research and 
development program improvement 
efforts 


1 r 
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Apaendix A 



alternate approaches to the 
Establishment of researcOnd development programs 



The identification and establishment of research and development 
programs by the Research and Development Section staff is essential for 
the successful implementation of the impact assessment system. In order 
to assist the Research and Development Sectioa staff in identifying 
research and developme/3t programs. Project IMPACT staff has identified five 
alternate approaches. These alternatives are presented on the following 
pages and^^^^ intended to be used as a basis for discussion among the 
Research and Development Section staff. The Project IMPACT Staff, 
recommends the first alternative as it is based on the current goals of 
the" Research and Development Section. 
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Alternative #1 
PROGRAMS BASED ON' FIVE MAJOR GOALS 



This alternative uses the five major goals identified by the 
Illinois State Board of Education/Department of Adult, Vocational 
and Technical Education in 1982 for the development of programs. 
These goals were developed as a result of a series of planning 
sessions held to identify key issues and solutions to some of the 
common problems addressed by the fourteen thrusts for the 80's. 

Research and Developmeht Programs Based on Five Major Goals 



Program 



Goal s 



Suggested year 
jfor ^olfg 
assessment 




1. Responsiveness 
to Individuals 



Economic 
Well-being 



3. Skill 

Development 



4^. Collaborative 
Efforts 



5. Support 
Services 



To provide access for youth and adults to 
more diverse vocational education programs ; 
which are suited to their needs, interests, ^ 
and abilities and which are consistent with • 
labor market needs. ; 

To promote vocational programs and services - 
which foster the economic well-being of i 
individuals, local communities, and the ' 
state. 

To support quality programs of vocationral 
and technical education which facilitate ; 
individual attainment of basic, technical , | 
and survival skills necessary for employment- 

To strengthen vocational education programs 
through collaborative efforts among and 
between job training providers and business, 
industry, labor, government and community 
agencies. 

To extend support services that assist youth 
and adults in vocational education programs 
to succeed- in the* work force. 



FY 1985 



FY 1986 



FY 1987 



Strengths: 

- Based on a refinement of the 14 thrusts from input by State Board staff, 
Illinois Council of Local Administrators and LEA's. 

- Major goal's are already identified an*d issues identified may be used as 
a basis for objectives and funding of projects. 

- Allows for cTssessing impact of one program per year, within a reasonable 
• time. 

Weaknesses: 

- Currently funded projects may not conveniently fit into program areas. 
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Alternative J 2 



COMBINE SIMILAR THRUSTS INTO RESEARCH AND DEVELOPMENT PROGRAMS 



This alternative uses the 14 thrusts as the basis for the development 
of programs. By combining similar thrusts, five programs are created. 
One program would be assessed each year so that all programs would be 
Assessed every five years. 



Research and" Development Programs Based on Combinations of Thrusts 





Program 


1 
1 

! 
! 

' Subprogram (thrust) 
4 


Number of 
related 
projects 
funded 
FY 81 FY 82 


ouggesteci ypar 
for long term 
assessment 


1. 


Strengthening Voc Ed^ 
Capacity 


Research, Development 
i Dissemination 
' Planning and Evaluation 
i Voc Ed Staff Development 


OA 

5 

i 5 


CO 

2 
5 


' FY '83 






j subtotal 


1 


30 




2. 


Responsiveness to 
Individual s 


1 Access & Equity 
! Individual Needs 
i Older Citizens 


i n - 

16 
1 


5 
10 
0 


FY '84 






i subtotal 

1 


28 


15 




3. 


Skill Development 


Basic Skill Development 
Survival Skills 
Competency Based 
subtotal 


4 
0 
22 
28 


■ 2 
0 
13 
15 


; FY '85 


4. 


Public Understanding 
and Policy 

* 


Pol icy & Legislative ' 

Development 
New Economic Concepts 
Public Awareness 


0 ' 

5 

2 


0 
3 
2 


FY .'86 

1 

! 






subtcrtal - 


7 


5 




5. 


Organizational Linkages 


Business/Industry/ 

Education Coordination 
Agency Coordination 


4 
2 


3 
2 


FY '87 






subtotal 


6 


5 





Strengths: 

- Based on the 14 thrusts, adding continuity to long term organizational commitments. 

- Reduces the 14 thrusts to five easier-to-understdnd research and development 
programs. This should facilitate the overall management of the research and 
development effort, 

- Allows for assessing impact of one program per year, within a reasonabi e. time. 

- Assesses irtpact of programs emphasized for a number of years, first, and allows 
time for' lesser funded programs to be further emphasried. 

Weaknesses: ^ - • 

- Subprogram goals may not always be related. 

- Adds dimension of "subprogram" (thrust) to the project-to-program conceptualization, 
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Alternative #3 
EACH OF 14 THRUSTS CONSIDERED A PROGRAM 

In this alternative each thrust becomes a program, creating 14 different 
programs. One to five programs would be assessed each year in order for 
all programs to be assessed every five ^ears. 



Program 



1. Research, Development, Dissemination 



1, Individual Needs 

3. Planning and Evaluation 



4. Competency Based 

5. Survival Skills 



6. Access and equity 

7. Vocational Education Staff Development 

8. Public Awareness 

9. Older Citizens 



10. New Economic Concepts 

11. Basic Skill Development 

12. Business/Industry/Education Coord, 

13. Agency Coordination 

14. Policy & Legislative Development 



subtotal 



subtotal 



subtotal 



subtotal 



subtotal 



Number of 
related 
projects 
FY 81 FY 82 



24 



23 



24 


23 


16 


10 


5 


2 


21 


12 


22 


13 


0 


0 


22 


13 


11 


5 


5 


5 


2 


2 


1 


0 


19 


12 



3 
2 
3 
2 
0 



10 



Suggested year 
for long term 
assessment 



FY '83 

FY '84 

FY '85 

FY '86 



FY '87 



Strengths : 

- Assess impact of each thrust, providing several projects are funded in each o"e 

- More even distribution of individual projects to choose from per year (approx. 20j, 

Weaknesses: 

- In some cases few or no projects to make up program, at least at present. 

- Too tnany programs to assess in FY '86 and FY '87. 

- Too many programs to fund with multiple projects in the near future. 
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Alternative # 4 

COMBINATIONS OF 11 PROGRAMS CURRENTLY IDENTIFIED 
BY RESEARCH AND DEVELOPMENT SECTION 



This alternative combines the eleven subject specific programs 
currently used by the Research and Development Section into six 
programs* One or two programs would be assessed each year so 
that all programs^can be assessed every five years. 

Research and Development Programs Based on Combinations of Current Programs 



Programs 



Number of 
related projects 
currently funded 
^FY 81 FY 82 



Suggested year 
for long term 
assessment 



1. All Vocational Education 



2. Agriculture 

3. Health Occupations Education 



♦4. Home Economics Education 
Career Education 



5. Business Office/Distributive/ 
Cooperative Education 



6. Industrial Arts/Technical /Trade/ 
Industrial Education 



subtotal 



subtotal 



subtotal 



74 
74 

3 

2 



56 



1 
1 



4 

0 
4 



11 



3 
0 



subtotal 11 



subtotal 7 



FY '83 

FY '84 

FY '85 

FY '86 

FY '87 



Strengths : 

- No more than 2 programs assessed each year. 
Weaknesses : 

- Bulk of projects identified as being "all vocational education". 

- Too few projects contributing to, programs in most cases. 

- The 14 Thrusts are not easily, or even reasonably categorized into 
programs defined this way. 
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Alternative #5 



UNIVERSITY RESPONSIBILITY FOR RESEARCH AND DEVELOPMENT ON THRUSTS 



This alternative assjgns the responsibil ity" of research and development 
efforts, categorized by thrusts^ to the 9 universities which are a part 
of the liaison council. 

Research and Development Programs Based on Thrusts Assigned to Programs 



University 



Thrust 



Num^ef' of 
projects • 
FY 81 FY 82 



Year of 
assessment 



SIU-C 



U of I 



ISU 
SIU-E 



WIU 
NIU 
EIU 



CSU 



Competency Based 
Survival Skills 



Research^ Developm^t, 

Oissemihation 
Policy and Leaislative. 

Development ' 



Individual Needs 
Older Citizens 
Business/ Industry/Education 
Coordijiation 



Access and Equity 
Planning and Eval uatlon* 
Vocational Education Staff 
Development 



Publ ic Awareness 
New Economic Concepts 
Basic Skill Developm.ent 
Agency Coordination 



22 
Q 



7 



13 
0 



subtotal 



subtotal 



subtotal 



subtotal 
subtotal 



22 


1 






0 


0 


24 


23 


16 




. 1 




4 . 




21.' " 


13 


11 


5 


5 


. 2 


5- 


5" 


21 • ' 


12 


2:" ■ 


2 • 


•5 


3 ■ 


4 


2 


2 . 


2 


13 


9 



FY '83 



FY 



FY '85 



FY '86 



FY .'87 



Strengths : , . ^ . 

- Each university's research and development could be viewed as a research and 
development program. • ^ 

- Responsibilities on universities for long term pursuit of particular thrusts. 

- Research and development efforts spread geographically across state. 

- Builds upon the, 14 thrusts. ^ ^ 



Weaknesses : 



9. 



- Staff capacities arid interests at universities will 1 lively be broader than 
particular 3s;^igned thrusts. . - f 

- May be build in bias on part of evaluators. 

* Decentralized approach could lead' to haphazard,, or at leas.t uneveri, progress 
O toward each thrust. ' 
EgJC" Would be,pollticrlTy upwleldy, if not substantively. 
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APPENDIX B 
A SAMPLE PROGRAM SPECIFIC IMPACT SURVEY 
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INTRODUCTION 



(A Sample Program Specific Impact Survey) 



A sample Program Specific Impact Survey has been developed and 
included here to illustrate the types of questions which might be asked 
to assess the impact of a hypothetical research and development program, 
Responsiveness to Individuals. Tffe program was developed to provide a 
basis for development of the survey. 

The sample program is based on alternative 2 (Appendix A) presented 
by the ^mpact Project staff as one of several possible approaches which 
could be used to establish research and development programs. The 
sample program consists of twelve research and development projects 
which were funded for fiscal year 1982 which have related goals, whose 
goals relate to the program's objectives and whose intended impacts 
contribute to the sample program's intended impacts. 

"^To aid the reader *in understanding the materials presented here. 



background is provided by bringing together information related to the 
development, implementation, and assessment of research and development 
programs which appears in various sections of the Impact Assessment 



System^ocumeht. The purpose is to provide a coherent view of a research 
and development program and to present one means of assessing the impact 
of such programs, the Prograir: Specific Inpact Survey. 
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BACKGROUND 

\ 

\ 

Program Development : ' The research and develoBment program is a group of 
program improvement projects funded by the Research and Development 
Section that are seen by the Research and Development Staff as having 
related goals. The program would therefore be developed based on a 
broad need area. The Impact Project staff has suggested several 
alternatives for developing R&D programs as follows: 

• Develop programs for each of the 14 thrusts 

• Develop programs by combining similar thrusts 

• Develop programs by using combinations of the 11 programs 
currently identified by the R & D Section 

• Develop programs by assigning related thrusts to universities 
Regardless of how programs are developed, by the above or other 

methods, the programs should be developed before the issuance of , 

Requests for Proposals. 

Program Goals : Once the programs are identified, a program goal(s) 

should be developed for each program. Program goals ^re what the state 

education agency (Research and Development Section) would like to achieve 

as a result of funding^e or more program improvement projects within 

each program. \ 

Program Objectives ; Program objectives are the outcomes which will lead 
^ 

to achieving the identified goal(s) of a program. The objectives should 
be general izable to a number of different projects. * ^ 

Program Procedures ; The* program procedures will relate to the RFP process 
in most. instances^ However, it is not necessary that funding projects to 
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alleviate a need(s) is the only procedure which may serve to address a 
program objective. For example, the R&D Section may need to cooperate/ 



collaborate with other offices, departments or sections at the state 
level to accomplish the goals and objectives of a particular program. 
The degree to which'procedures will be related to the RFP process will 
depend on whether the procedures require the R&D Section to act alone, 
whether the R&D section will delegate the required action to another 
agency through the RFP process, or whether the action needed to accom- ^ 
plish program objectives requires joint action of the R&D Section and 
others (i.e. University R&D staff, other state level agencies, local 
level personnel). 

Intended Prpgram Impact :' Intended program impact or what is expected to 

be the consequences of implementing each R&D program should be planned'. 

The intended impact statements should specify the target population(s) 

expected to be affected and identify the precise nature of the intended 

consequences. Ideally, they should have an impact standard against 

which the actual impact achieved can be measured. For example, in the 

program, Responsivenes^ to Individuals, an intended program impact 

statement might be as follows; » 

Increase by twenty-five percent the number of special 
needs secondary s'.wients in Illinois who complete 
vocational education programs in new and emerging ^ • " 
occupa^tions which have high employment demand as 
compared to the 1982 figures. 

When developing intended program impact statements, the developers 
should be realistic as to the extent of impact which can be expected 
based on the procedures to be used to achieve the impact. 
The Relationship of Projects to Programs : Projects will be linked to 



a program by their goals relating directly to a program's objectives. - 
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The program objectives should serve as a basis for guiding the development 
of project goals and the intended impact{s) of projects should contribute 
to the achievement of intended program impact. 

This process implies the need for clearly communicating the program 
and its components to potential proposal writers and to project directors 
of funded projects. It also would require very careful development or 
revision of project proposals to assure the goals of projects relate 
directly to program objecti>4s and that their intended impact will con- 
tribute to intended program impact. Further, it is conceivable that the 
degree to which projects can address program objectives may require 
program revision or extension (in time) to achieve the goals, objectives, 
and intended impact of a' program. 
Measur";nc) Program Impact : One way program impact can be measured is through 
a program specific impact survey. The program specific impact survey should 
be developed based upon the information gaifffJNduring the in-depth investi- 
gations of long term program impact and upon evidence of actual impacts. 
The purpose of the survey is to gather additional impact data to validate 
the findings from the on-site visits of the long term impact assessment. 
For example, during an on-site visit one might find that a project's staff 
appeared to spend (or to have spent) considerable time and effort providing 
technical assistance to others interested in their project's outputs. 
From interviews and documents at the site, information can be acquired for 
including questions in the program specific impact survey for validating 
the provision of technical assistance and identifying persons who should 
receive the survey. ^ 

Target audiences for the survey . The program specific impact survey 
should be administered to impl em.6ntatjon/util ization sites external to the 
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project sites included in the long term impact assessment sample for a' 
specific program. Therefore, the survey will have as its target audience, 
sites and persons who have implemented/utilized products of one or more of 
the projects within a research and development program being assessed. 
The target audiences identified in projects' dissemination planning outlines 
and impact statements should be represented in the target audience for the 
program specific impact survey. 



PROGRAM: Responsiveness to Individuals 



PROGRAM-£OAL: To provide access to youth and adults, to more diverse vocational 
education programs which are suited to their needs, interest and 
abilities and which are consistent with labor market needs. 

Subprogram (thrust): Access and Equity, Individual Needs, Older Citizens 



Program Objectives 



To increase the availability of 
vocational education programs 
in the state whicTi have the 
capacity to meet the needs, 
interests, and abilities of 
youth and adults in areas of 
employment demand. ^ 



To increase and improve voca- 
tional education programs and 
services for special popula- 
tion groups. 



Program Procedures 



1. To fund through the RFP process, 
research and development projects 
which propose development, testing 

(and/or refinement of vocational 
education programs, or products 
which will increasei^the capacity 
of vocational education programs 
to meet the needs, interests and 
abilities of youth and adults in 
areas of employment demand. 

2. To fund through the RFP process, 
research and development projects 
which propose development, testing 
and diss^emination of vocational 
education programs, program 
models, products and/or orher out- 
puts for improving the delivery 

of vocational education to special 
population groups including*: 

Limited English Proficiency 

Disadvantaged 

Handicapped 

Native Americans 

Displaced Homemakers 



ERIC 



Intended Program Impact 



2. 



Increased availability of voca- 
tional education . programs in the 
state which have the capacity to 
meet the needs, interests, and 
abilities of youth and adults in 
areas of employment demand. 



7 b 



Increased access and equity which 
result in increased participation 
of special population groups par- 
ticipating in vocational education 
programs, acquiring marketable 
skills and finding employment. 

Increased modification of voca- 
tional education programs to ac- 
commodate special- population 
groups.^ ' _ 

Improved planning and implemen- 
tation of vocational education 
programs designed to serve 
special population groups. 

Improved supportive services for 
special population groups par- 
ticipating in vocational educa- 
tion program's. 

continued 
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Program Objectives 


Program Procedures 


Intended Program Impact 


3. To increase and improve the 
resources and assistance 
available for the estab- 
lishment, maintenance and 
extensions of vocational 
programs/activities at 

the local level in secon- 
dary and post-secondary 
educational institutions 
for serving and/or pre- 
paring personnel to serve 
i special population groups. 

4. To increase cooperation and 
coordination between voca- 
tional education and other 
state/local agencies serv- 
ing special population 
groups. 


3. Same as objective 2 

4. To fund through the RFP process, 
projects which propose to develop, 
test and disseminate to vocational 
educators programs, program models, 
products or other research and 
development outputs for the purpose 
of improving cooperation and col- 
laboration between vocational edu- 
cation and other state/local agen- 
cies which services to special 
population groups. 


Increased knowledge and aware- 
ness by vocational educators, 
of the varied assessment, and 
placement procedures that can be 
utilized with special population 
groups in vocational education 
programs at the secondary, post- 
secoBd^ry and adult levels. 

3. Greater availability and famili- 
arity by vocational educators of 
models, resources and strategies 
for serving special population 
groups. 

Increased numbers of vocational 
educators will be trained in 
methods and resources appropriate 
for improving vocational training 
for spec^l population groups. 

Improved preservice and inservice 
teacher education programs for 
working with special population 
groups. 

4. Increased cooperation and col- 
laboration between vocational 
education and other state/local 
agencies providing services to 
special population groups. 
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Program: Responsiveness to Individuals 



- 


^ X 

Projects in Program 


Goals of Projects 


Intended Impact of Project 


1. Truman Bilingual Vocational 
Center - Project Mainstream 


To provide an intensive system of 
bilingual support services. for 
vocational education students who 
have limited English proficiency. 

10 OTTer prog ram b i ri in yri uciiiaiiu 
vocational education areas for 
limited English proficiency students. 

To develop and disseminate bil ingual 
and VESL materials for vocational 
education programs. 


Korean and Hispanic LEP adults in 
Chicago will acquire marketable 
skills and improve their ability to 
communicate in English. 

The materials developed will aid 
other vocational instructors in the 
state to develop suitable voca- 
tional materials for LEP populations.. 


2. Bilingual Vocational Education 
Project 


To provide assistance and resources 
to vocational programs to improve 
services to LEP students. 

- 


Greater familiarity by educators of 
models, resources, and strategies 
for serving LEP persons. 

BVEP client use of new material, 
strategy or information in develop- 
ing or implementing services for 
LEP persons. 

Greater availability and accessi- 
bility of instructional and resource 
materials to serve LEP students and 
assist vocational programs. 

Increased communication between 
vocational educators and ESL/ 
bilingual educators, among LEP 
programs,. and among projects and 
agencies providing, support services 
to LEPs. 


3. Development of a Preservice 
Training^ Program for Voca- 
tional Teachers of the 
Z') Limited English Speaking. 


To develop and disseminate resources to 
improve preservice and inservice vocar 
tional teacher training programs for 
teachers of limited English proficiency 
studjents. 


Teaching populations will be trained 
in methods and resources appropriate 
for the LES vocational student. 



Projects in Prdgram 



)__ 

Goals of Projects 



Intended Impact of Project 



4. A plan to Develop and Compare 
Two Vocational EdiKation 
Programs for Limited English 
Proficiency Students.- 



To analjb^ze the advantages and dis- 
advantages of an English core language 
vocational instructional model in ^ 
terms 'of adoptabil tty, "teaching 
effectiven^ar?^ ^anguage^and vocational 
skill acquisition, and student com- 
p\etion, performance and attitudes. 

To .compare the English, core language 
vocational instructional model with 
th£ bilingual vocational instruct4-OM4— 
J>model implemented and analyzed in FY 81 



As a result of this project, 
administrators and instructors 
on state and local levels will 
have access to data and inst- 
ructional materials which will* 
greatly aid them in planning, 
implemeating and/or improving 
their vocational training 
programs. 



5. ' Project Access 



To provide support services to LEP ^ 
students. 

To develop and disseminate materials , 
for use with LEP students enroll led in 
vocational education. ^ 

To provide assistance to vocational 
educators in Illinois high schools 
to improve vocational education for 
LEP students.' 



An increased number of LEP 
students enrolled in voca- 
tional education courses. 

Improved services to LEP 
students enrolled in vocational 
education courses. 

An increased number of LEP 
students with marketabl e* job 
sJ<ills which increase their 
opportuntties for entry lev-eK,*, 
jobs after high school . 

Iricreased opporfun 
college'4ftid profess 
levels for LEP stude 




6. Research, Evaluation and 
. Program Improvement for 
Limited English Proficiency 
Students in Vocational 
Education: A continuation 
Proposal, 1981 -82. 



P 



To provide ISBE/DAVTE with the capacity 
to strengthen vocational education per- 
sonnel at the secpndary, post-seconda/ry 
and adult levels who serve LEP youth 
and adults b^^^TJr^Qviding resou^ce 
materials ftfr adnmHstrators, teachers, 
counselors, » and teaclper educators*. 




Increased knawl edge ^fifl ^re- 
ness^of the varied identiri- 
cation, assessment, ai/d place- 
ment procedures that can be 
utilized for LEP students in 
cational education programs 
at the s^ndary, post-secondary 
and adult55evels, 

continued 



Projects in Program 


Goals of Projects 


Intended Impact of Project^ 




• 


Improved design of vocational 
education programs which meet 
the special needs of limited 
English speaking persons at 
the secondary, post secondary 
and adult levels. 

Improved support services for 
vocational LEP students , 

Increased knowledge and aware- 
ness of the varied support 
services and resources that 
can be utili2(|pl and obtained 
for LEP students in vocational 
education programs. 


7, Development of multi-cultural 
competency-based vocational/ 
technical curricula- 
Phase III, 

• 


To make vocational/technical training 
effectively available to the under- 
served Americans whose limited English 
abilities prevent them from acquiring 
necessary skills and krfowledge related 
to suitable vocations and careers in 
the mainstream of national development. 

s 

V 

1 

V 

« 


The integration of a VESL train- 
ing component into the present 
Career Development Center program 
and establishment of a coopera- 
t^'ve agreement with LESA agencies 
providing training intakes which 
will result in an improved ' 
capability to provide multi- 
cultural, competency-based 
vocational training. 

Availability of resources for 
further expansion of traintng 
opportunities to limited English, 
speaking adults which can be 
adapted/adopted in local, 
regional or national settings. 

Increased bilingual and multi- 
LU 1 uura 1 1 n b truL L 1 una 1 
capabilities of CDC vocational 
educators. 



r 

Projects in Program 


Goals of Projects 


Intended Impact of Project 


8. Surmounting Architectural 
Barriers to the Handicapped 
in Vocational Education - 
Phase III. 

» 

< 


To make available to vocational 
educators resources for making 
vocational education programs 
accessible to the physically 
^ handicapped.' ^ 


Greater participation of physically 
handicapped persons in vocational 
education programs. 

Increased modification of voca- 
tional education laboratories and 
equipment at the local level to 
accommodate physically handicapped 
' students, 

Increas^4*.^©9Sieration between voca- 
tional^and specVl education per- 
sonnel in modifying laboratories 
and equipment for the vocational 
education and employment of the 
disabled. 

Improved career counseling of 
disabled students based on 
better assessment of the capa- 
bilities and possibilities for 
training and employment^ of the 
disabled. 

Inclusion of content that stres- 
ses accessibility for the 
physically handicapped in teacher 
education programs in vocational 
and special education. 


9. Improving Services to 
Disadvantaged and 
Handicapped Students 
in Vocational 

' ^ Education. 

* 


To initiate a system in which 
disadvantaged, handicapped and 
limited English proficiency students 
are provided a comprehensive deliv- 
ery of vocational services and 
supportive resources necessary to ^ 
insure their successful partici- 
pation »in training level programs 
and/or job placement. 


Increased access to vocational 
education programs for disadvan- 
taged, handicapped, and limited 
English proficiency students. 

Increased and improved supportive 
services for disadvantaged, 
handicapped and limited English 
proficiency students . 

continued 
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\ Projects in Program 


Goals of Projects 


Intended Impact of Project 




To establish a reciprocal network 
system^ among public and private 
agencies to provide for the trans-' 
luion trom scnooi to une worK wonu 
and/or community of disadvantaged, 
handicapped and limited English 
proficiency students. 


Increased aVciilability of re- 
source material s in vocational 
program areas for limited English ^ 

Improved attitudes of limited 
English proficiency students. 


10. Technical Assistance and 
Dissemination Network: 
Illinois Special Needs 
Populations. 


To provide technical assistance 
and consultant services to local 
education agencies concerning the 
improvement of vocational education 
for special populations. 

To continue Dissemination Center 
services through materials dissemi- 
nation, impact evaluation, technical 
assistance, materials development 
and newsletter communication at the 
state, local and university level. 


Increased availability of re- 
source ma.terials in vocational 
education for. serving special 
po pulations. 

Increased and improved access 

and equity for special popula- \ 

tions in vocational educa-^ion. \ 

Improved dissemination of ideas, 1 
materials, resources and services } 
to vocational educators serving' 1 
special populations. 

Increased efficiency and accounta- 

Kilifw n -f unrafinnal pHiirafnr^? in 

individualizing vocational' educa- j 
tion to accommodate special ^ j 
populations. ^ \ 


11. Programmatic Emphasis in 
Vocational Special Needs 
Education : Continuation 
Proposal 1981-82. 


• 

To develop and expand preservice 
andjnservice training opportuni- 
ties' for persons involved in 
vocational education for ^special 

.needs learners. 

T9^-*4£iJjtate linkages between 
vocational education, special 
education, and vocational re- 
habilitation at the university, 
state education agency and local 
level s. 


Increased numbers of undergraduate 
and^ graduate students (at UIUC) re^ 
ceiving instruction in vocational 
education for^special needs 
learners. 

Increased availability of off- 
campus courses and other forms of 
inservice training and technical 
assistance for personnel working 
with special needs students in 
yocatfonal education., 

continued 
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Projects in Program Goals of Projects 



12. Institute for Native To provide assistance and resources 

American Development to native Americans and to educa- 

tors serving tHfem, to increase 
their participation in higher 
education programs including 
vocational education. 

To provfde a system of support 
services for native Americans who 
enroll in vocational education 
programs. 

To develop a model for vocational . 
training and support of native 
^Americans within higher' education , 
which is transferrabl e to other 
vocational training sites and 
levels. 




\ 



/ 




'I 

Irvtended Impact^of Project 



An expanded knowledge base for 
planning, implementing, and 
evaluating inservice/preservice* 
programs in Illinois. 

Increased opportunities for special 
needs learners to participate in 
vocational education programs that 
are appropriate for and responsive 
to their educational an<)t employ- 
ment needs. /-v^ ^ 

Availability of a successfu\ model 
for vocational training and 
support of native Americans within 
an institute of higher education. 

Availability of a self-development/ 
goal setting curriculum for use 
with native American and other 
special populations in vocational 
programs at the postsecondary 
level • 

Irtarease tj>e numbers of native 
Ame?Tc^m^-^eing served in post- 
secondary vocational education 
programs. 
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SAMPLE PROGRAM SPECIFIC IMPACT 
SURVEY 



code 



The purpose of .this survey is to gather data for use in assessing 
the long term impact of the research and development program 
Responsiveness to Individuals, 

This research and development program o^ the Illinois State Board 
of Educatiop, Department of Adult, Vocational and Technica»l Education, 
was composed of twelve research and development projects which are 
listed on page 2. 

I 

You have been identified as a participant, i.e., staff member of • 
a project, workshop participant, benefactor of technical assistance, 
or user of a project's products or services in one or more of the 
project^^ within this program. As a participant in one or more of the 
projects you are requested to complete and 'returjj^this Program 
Specific Impact Survey, ' ^ 

Th/ Survey form is coded; however, all respondents and responses 
are confidential. The survey does have several respondent groups; 
therefore you are requested to identify the respondent group to 
which you belong. Please place the letter which identifies the 
respondent group to whicfn you jL)elon§ in the square in the lower ^ 
right corner af- this page. 
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Respondent Groups 



A. 
B. 
C. 
D. 
E. 
F. 
G. 
H. 
I. 



Bil inqual/ESL~ Program Director 
Bilingual /ESL Teacher 
Community Agency Staff Member 
Curriculum Developer 
Educational Consultant 
Guidance and Counseling 
Non-Vocational Teacher 
Project Director 
Project Staff Member 



J, 
K. 



st-Secondary Voc. Teacher 
School Admini strator 
Secondary Voc. Teacher 
Special Education Teacher • 
Staff Develdfpment Super- 
visor 

Vocational Director ^ 
Other, please specify 



respondent 
code 



ERIC 



9 



3, 



'9 



— » 

PROGRAM 



71 
Part I 

RESPONSIVENESS TO INDIVIDUALS 



Research and Development Projects Included in thfis Program 

1 



!• Truman Bilingual Vocational 
, Center: - Project Mainstream 
Truman College 
Chicago, Illinois 

Project Director: John 
Gianopulo-s 

2. Bilingual Vocational Education 
. Project 

Northwest Educational Cooperative 
. Arlington Heights, Illinois 
Project Director: Jeanne 
Lopez Valadez 

3, Development 'of a Preservice 

Training Program for Voca- 
^ tional Teachers of the 
^ limited English Speaking 
Illinois S$ate University 
Normal, Illinois 

Project Director: Ms. 
Margaret Kang 

4i A Plan to Develop and Compare Two 
Vdcational Educatit)n Programs for 
Limited English Proficiency 
Students 
V/aubonsee Community College 
Sugar Grove, 111 inois 

Project Director: Michael 
Kelly 

5. Project Access/ 

Benito Juarez '/tligh School 
Chicago, Illi/lois 

Project D/rector: Jose A. 

Rodrique^ 

6. Research E^al uation^ and Prpgram 

Improvement for Limited English 
Proficiency Students in 
Vocational Education 
University of Illinois 
Champaign, Illinois 

Project Director: Dr. L. Allen 
Phel ps 



7. 



8. 



9. 



10. 



11 



12. 



Development of Multi-Cultural Com- 
petency-Based Vocational/ 
Technical Curricula - Phase III 

Southern Illinois University 

Carbondale, Illinois 

Project Director: larry Hepburn 

Surmounting Architectural Barriers 
to the Handicapped in Vocational 
Education - Phase III 
Northern Jll inois University 
Dekalb, fll inois 

Project Director: Dr. Thomas 
Erekson 

Improving Services 'to Di sadvaj;ltaged 
and Handicapped Students in Voca- 
tional Education 

Board of Education, City of Chicago 
\Chicago, 111 inois 

Project Director: Marjorie Peqo 

Technical Assistance and Dissemi- 
nation Network: Illinois Special 
Needs Populations 
Illinois State University 
Normal Illinois 

Project Director: Catherine 
Batsche 

Programmatic Emphasis in Vocational 
Special Needs Education 
University of Illinois 
Champaign, Illinois. 

Project Director: Dr. L. Allen 

Phelps 

Institute for Native American 
. Development 
Truman !Co11 ege 
Chicago, Illinois 

Project Director: Ms. Dorene P. 

Wiese ' — . 

? 



J 



) 
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Identify by circling the project number(s) below, the project(s) in 
^Which you ha.ve been a participant. 

1 2 3 4 5 6 7^ 910 11 



12 
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SERVICES OF RESEARCH AND DEVELOPMENT PROJECTS 



♦ 

I 

t 

i 

1 

! 

i 
i 
1 

i 

1 
j 

1 Research and Development Projects 


Conference 


Coordination 


Course Development 


Dissemination of Products . 


Field Testing 


InserVice Training 


Materials Development 


Pre-Service Training 


Supportive Services for native 
lAmerican Students 


Supportive Services for 
Disadvantaged Students 


Supportive Services for 
Handitapped Students 


Supportive Services for 
Limited English Proficiency 


Technical Assistance 


Workshops 


1. Project Mainstream Truman 
Bilingual Vocational Center 














X 




/ 






. X 






2. Bil inguaUVocational 
Education Project 




X 




X 


X 




X 




t 








X 




3. Development of a Preservice 
Training Program for 
Vocational Teachers of the*"-. 
Limited English Speaking 










X 


X 


X 






> 










4. A Plan to Develop and Com- 
* pare Tv;o Vocational Educa- 
tion Programs for Limited 
English Proficiency Students 


• 

X 










X 


X 


X 








X 






5. Project Access 












X 


X 
















6. Research, Evaluation^ and Pro- 
gram Improvement for Limited 
English Proficiency Students 
in Vocational Education 










X 


X 


X 














/ 

/ 


7-^ Development* of Multi- ' 

Cultural Competency Based Vo- 
cational /Technical Curricula 










X 




X 












k 


X 


8. Surmounting Archltectual Bar- 
riers to the Ha^ndicapped in 
Vocational Edo&itlon 












X 


X 












X 




9. Improving Services to Dis- 
advantaged and Handicapped 
otuoents in vocaLionai lu. 












X 


X 






X 


X 








10. Tefchnical Assistance & Dis- 
semination Network; 111. 
Special Needs* Populations • 








X 






X 












X 


X 


11. Programmatic Emphasis in Voc. 
Special Needs Education 






X 






X 


Y 












X 




12. Institute for Native 

American Development i 












X 


X 




X 













RrSEARCH m OEVELOPHENT PROJECTS PRODUCT USE SHEET 



Projects 


1 

» 

Products 


i 

*-> 

1 
1 


Received Product 


°? 
li 

li 

St 
5.3 


Have Not Used Product 


Plan to Use Product 


Do Not Plan to Use 
Product 


u 

CI 

o 

L. 
Q. 


/ 

Productsi 


1 


Received Product 


Have Used and/or 
Currently Using 


Have Not Used Product 


Plan to Use Product 


Do Not Plan to Use 

Product 


Project liainstredm 
Truman College Bilingual 
Vocational Center 


Bilingual Supportive Services Model 
for Vocational Education 














Project Access 
Benito Juarez 
High School 


Vocational Curriculum Materials 

for I EP Students J * 


P 17 












Bilingual Curricula: 

- Autobody Repair \ 


p i 












Video-tape: Profile of an 
LEP 


.p 18> 












- Accounting 

- Data Entry 


Pi 












A-/ Presentation: Guidance 
and Counseling of LEP*s 


p id 










• 


- Key Punch 


P 6 












- Drafting 
Bilingual Project Administrators 


P 7 






if 






Research, Evaluation 
and Program Improvement 
for LEP Students in 
Vocational Education 


Handbook on Identification. 
Assessment and|Placenent of 
LEP Students* 


p 20 
















• 










1 1 1ngual 
oc. Ed. 
roject 


Program Development Module 
Staff Traininq Module 


P 8 
P 9 












Handbook on Instructional 
Strategies and Support Services, 
for LEP Students 


P 21 












Materials Adaptation Module 


p 10 












Preservice Training 8 
Program for Vor. V 
Teachers of Limited P 
English Speaking 


Videotapes - language Adaptations 


P 












Ins'ervice -Resource Guide 
• 


P 22 




f 








learninq Modules 


P 12 










^ 


/ ' 














Development of Multi -Cultural 
Competency-Based Vocational 
Technical Curricula 


VESL Guidelines 


p 23 












A Plan to Develop and 1 
Compare Two Voc. Ed. J 
Programs for LEP 
Students • 1 


i ■ 

English Core Language Instructional Model 


p 13 












Thai Version #f Lexicons 


p 24 












Program Introductiv and 
Outlines f 


p 25 












Bilingual Instructional Model 


p 1*4 












Instructional Materials: 
- Machine TooIt^ 


P ^5 












Introductory Scripts/Audio Tapes: 
- Spanish 


p 26 












- Secretarial Scienc* 


p 16 












-.Thai • 

'-^ . 




























' V 















1.*) Directions: *0n this page and the following page, products developed and/or disseminated by Research and Development 

( projects within this program are listed. For each project in which ^ou have been a participant, put an 

) -X- Jn the Appropriate square to indicate usage of prodjcts you have received. Complete only for products 

/ * you r>»(iJ.ka^i lear^ ill others blank. 
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Projects 


1 
> 

Products 


Product Code 


Received Product 


Have Usjed and/or^ 
Currently Using 


Have Not Used Product 


Plan to Use Product 


^Oo Not Plan to Use 
Product 


Projects 


/ 

Products! 


«» 

Q 

Z 

o. 


Received Product 


Have Used and/or 
Currently Using 


Have Not Used Product 


Plan to Use Product 


Oo Not PUn to Use 
Product 


Project ria1nstre«m 
Truman College Bilingual 
Vocational Center 


Bilingual Supportive Services Model 
for Vocational Education 


P 1 












■ 1 

Project Access 
Benito Juarez 
High School 


t , 

Vocational Curriculum Materials 

for LEP Stud^nt^ J * 


P 17 












Bilingual Curricula: 

- Autobody Repair y 


A 

p J 












Video-tape; Profile of an 
LEP 


.p 18^ 


> 










- Accounting 

- D^ta Entry 














A-V Presentation: Guidance 
and Counsel ing of LEP's 


p t9 










• 




P 6 












• Drafting 
Bilingual Project Administrators 


P 7 
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Research, Evaluation 
and Program Improvement 
for LEP Students In 
Vocational Educat Ion 


Handbook on Identification, 
Assessment and ^Placement of 
LEP Students* 


p 20 
















• 










Bi 1 ingual 
Voc. Ed. 
Project 


Program Development Module 
Staff Training Module 


n R 
P *5 

D 9 












Handbook on Instructional 
Strategies and Support Services, 
for LEP Students 


P 21 












Materials Adaptation Module 


p 10 


























Inservlce -ilesource Guide 
• 


P 22 
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Preservice Training 
Program for Vor. 
teachers of Limited 
English Speaking 


Videotapes * Language Adaptations 


P 11 












Learning Modules 


P 12 










^ 


/ ' 














Development of Multi-Cultural 
Competency-Based Vocational 
Technical Curricula 


VESL Guidelines 


P 23 












A Plan to Develop and 
Compare Two Voc. Ed. J 
Programs for LEP 
Students ' 1 


; ■ 

English Core Language Instructional Model 


p 13 












Thai Version #f Lexicons 


P 24 












Program Introducti(tf and 
Outlines P 
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Bilingual Instructional Model 
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Directions: 'On this page and the following page, products developed and/or disseminated by Research and Oevelopnent 
projects within this program are listed. For each project In which you have been a participant, put an 
" jn the appropriate square to indicate usage of prodjcts you have received. Complete only for products 
u r>i;>e.i.ka#i lear^^ all others blank. 
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Fact Sheet on Disadvantaged and 
Handicapped Services 
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Project Managers Handbook 
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Newsletter; The Network News 
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2. Old you receive a product(s) (r.e:, hajjdbook, curH-CMlUm guide, 
resource material, etc*.) disseminated /by any of the' research and 
development projeets''in this, program? 

no . r [skip to/ question 4a | . • . .1 
. • * * . yes ^. 2 

3. «sing the Projects Products Use SHeei,* for^ each product ycJu are 
using, or have used, n'rtdtcate the extent of y,se by placing th^ 
product code in the blank(s) b6side'-the appropriate sta'tement(s) 
belQw. 



Prod^ict Number(s) . ' Extent of Use 



0 



t ' Total use.j have adopted the prdcluat without 

^ ~ changes for use in my^worK. . * ^ 



• » 

. Partial Use , ,1 havfe adopted pr adapted parts 
of the..product for use in my wo^k. 

Major Resource Use . - I freqjuently use the- 
product as a resource materi^ in my work. 

Minor Resource Use , I occasiowlly use the 
•product as a resource materia? in my work. 

T^h^ product js seldom used. 



' 4a.*^sing the Projects Product Use Sheet, rev\ew the. produces. 1 isted.^ 
Are there products listed which^u would have used i f'recetved?* 



(10 /. [skip to duestion 11 k .i . . 1 



yes. 



Z 



^b.^ Lf yes, identify each product by .its product codfe. ' 



. • • • .■ ' = 

4c. For the product(s) identified in question 4b antl/or for .pro<iucts' 
received but 'not used, estimate how many p^opl^ WAuld be affected 
.if you were to use the'product(s). ^ ^ ' •'N 



Number of Teachers 

Number of Students 

Number of Clients (p*ersons served 
other than students) . 

-Number of Parents \ 

umber of Administrc 



Nurj^er of Administrators 




O ,^ * ^ Number of Supervisow^ 
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5. If ybu have" received, but are not using a product(s) .deve-loped by a 
research and development project(s) within this, program, indicate the 
^reasons for not using the procfuct(s) by placing the^^roduct code -in the 
blank(s) to. th^ left of the apipropriate statement(s) which best describes, 
why you have not used the product. 



Product Number(s) 



Reasons for Not Using the Product 

The product does not fit my needs. 

Information and understanding is not adequate 
to use the product. ^ 

Lack su^orting materials to use the product. 

Lack equipment necessary to use the product. 

Time does not permit use of the product*. ^ 

Cannot accorranodate the product in my existing 
work structure (e.g. course content, rangg of , 
services, etc.) 

Lack financial resources to make changes needed 
to use the product. 

Lack support needed from others to be able to 
use the product. / 

Have b^en unable to get the training, technical 
\ass.istance or other help-needed to use the 
product. ' ^ 

Other (please specify) ' 



J\6. For the product(s) you are using what has been the effect of use? For 
^ each product ypu are using, indicate the effect of its use by product 
*cod^ in the blank(s) beside the following Appropriate Use Statement(s) 



Product Number(s) 



Effects of Product Use 

The product has improved my awareness and 
knowVedge of problems of special population 
groups (e.g., handicapped, disadvantaged, LEP, ^ 
etc.) in vocational .education. 

continued 




ERIC 



{J 



• V 77 • * . 



4 

\ 



ProduGt Number (s) ' 4 ftfects'of Product Use \. 

^ The product has improved my abilities to work^ 

with^ special population groups. 

The product has made it possible fo "^ncrease 

the participation of a special population 
group(s) in my vocational setting.^ * 

\ '.The product has helped. me to id^tify persons 

who can -help ^e address problems related to 
working with special population groups. 

The product his helped me to make. pj|ysical 
* ^ programmatic or other modifications in my 

setting to accommodate special population groups. 

^ The product has helped to improve the delivery of 

supportive services* to a special population group. 

\ ' The DToduct has provided me with a valuable 

curPPiiilum resource. 

'The product has helped me to adapt curriculum" ( 
and curriculum materials to meet the needs of a 
^ special population group. ' - 

The product has aided ia' the development of 

^ \ programs to meet the needs of a' special' 

population grou^p. . 

^he product has helped improve the identification, 

assessment and/or placement' procedures for a 
special population group. 

r "^The product has helped to improve staf ^development 

activitiels for serving, a special population group. 

The fJi^uct has helped to increase and' ijpprove 

^ Vocational competencies of special population 

group.^ y ^: , ' 

^ The product *<S[s madexit possible to increase the 

enrollment of a specia1>population group(s) in 
v'ocational education programs.^ 

For the product (s) you are using, estimate the number of people in eacli 
group (following page) who a^re affected by its use. 
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yProdute ' Product Group Affected by'u^ Product ' Product Product'^ 

) ^ Regular Students , ' . , 

Handicapped Students * ^ ' 

Limited Englis-h ' ' , ' , 

> Proficiency Students 



Oi.sadv^aged 'Students 

Native American 
Students 

\ Teachers 

, .Teacher Educators , 
... 1 

Parents 

Guidance CoiKise^loVs 
Administrators • 
Others (specify) 



8. Identify by product code how each product you received was acquired. 
Product Code ^ How Product Was Received 

' ' Workshop \ * 

J '.Mall (from Prb^t Staff) 

Principal 

Inservice 



Conference or Convention 



w 

^ Dissemination Agency Purchase (\B.g., ERIC or 

3^ Curriculum Publications Clearinghouse) ^ 

East Centpail Curriculum Materials Center 



technical Assistance Request 
From a. Colleague 

D^yJE Consultant ' ^ 

College or University Class • / 

Other, Please specify 
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9. Were you given instructions on how to use the product(s) you receiv^ 
through attendance at a workshop, inservice, conference, etc.? ^ 

^ no 1 

yes . . ; 2 

10a. Did you ever contac't a staff member of a-project(s) to help you 
understand and use the product(s) you received? 

no I skip to ^qu estion 11 | 1 

yes. . . . . ! 2 ' 

10b. If yes, explain briefly the nature of the contact(s) ' 



10c. ,If yes, what was the nature of the- project staff's* response? 

Assistance was provided in person by a project staff member 

Assistance was provided by phone by a project staff member 

Assistance was provided in writing by a project staff member 

/ Assistance was provided by a person designated by a project 
staff member 

Other, please specify ^ 



No .assistance was giVen ' 

11. As a participant in this research and development progran), did you contact 
a project(s) within this program for a service? 



no 

yes , 



Skip to and complete Part II| . .1 
2 



^If yes, what service(s) w^s requested? (Check those that apply) 

Requested help in planning a program. 

J ^ Requested" hel p in mdd.ffying a program. 

Requested help in implementfng a program. 



continued 



« 

80 



^Requested help in tdentifying and obtaining resources.* 

Requested he! fr in developing materials. 

Requested help in modifying materials. 

Requested help in eyarluating materials. 

Requested help with staff inservice. ^ 

Requested help in evaluating a program. 0 



Other, please specjify 



12. Did the project staff(s) respond to your request for services(s)? 



no 

yes, 



Skip to Part II and compl ete the SurveTT . 1 

— : . . 2 



If yes, vThat was the nature of the project staff's response? 

Assistance was provided in person by a project staff member. 

— - — f 

^ Assistance was provided by phone by a project staff member. 

^ ' Assistance was provided in writing by a/project staff member, 

Assistance was provided by a person designated by a* project 

staff member. 

Project ^staff were unable to provide assistance but referred 

•the request to an appropriate resource. 

No' assistance was given. 

Other, please specify 
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Part II 



As a participant in this research and development prograin, or as one 
having knowledge of the program, what overall effects do you believe / 
it has had on vocational, education in Illinois? ' 

I Circle all response codes you feel are appro^priate 1 -j^ 



ERLC 



The program has helped vocational and other educators become more 
sensitive to the needs^of special population groups 1 

The program has helped vocational and other educators to become 

more aware of the range of abilities of special population groups.../ 2 

The program has helped yocatio'nal educators design better 

programs to serve special population groups 3 

As a result of the program, more instructional programs 

specifically designed to serve special population groups have 

been implemented 4 

The program has made accessible to vocational educators needed 
resources for serving special population groups 5 

The program has increased communication among the many individuals, 
agencies ancj. groups serving Special populations * 6 

The program reduced the attrition rate of special population 

group members in vocational education 7 

'The program has increased the employment rate of members of special 
population groups completing vocational programs 8 

The progra/n has increased access and equity which has resulted 

in increased participation of special population groups in 

vocational education programs 9 

The program has motivated vocational educators to make academic 

and physical modifications in vocational programs to 

^commodate special population groups 10 

The program has improvecj supportive services for special population 
groups participating in vocational education programs 11 

The program has increased knowledge and awareness by vocational 

educators, of the varied assessment and placement procedures 

that can be utilized with special population groups in vocational 

education programs at the secondary, post-secondary and 

adult levels 12 

The program has increased the numbers of vocational educators 

trained in methods and resources appropriate for working 

with special population groups 13 
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The program has improved preservice and inservice teacher 

education programs for teachers of special population groups 14 

The program has increased cooperation and collaboration between 

vocational education and other state/local agencies providing 

services to special population groups 15 

The materials developed byithis program have aided vocational 

educators to develop suitajblve materials for serving special 

popul at i/On .groups which/meet their local level needs 16 

■ \ 

My responses to Part II are based on participant involvement Q 
My responses to part II are based on knowledge of the program Q 



Thank you for completing this survey form. Please re-insert 
the Product Usage Sheet and Return the survey in the 
envelope provided . 
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